How do individual cognitive differences relate to acceptability judgments ? A reply

Sprouse, Wagers, & Phillips (in press) carried out two experiments in which they measured individual differences in memory to test processing accounts of island effects. They found that these individual differences failed to predict the magnitude of island effects and construe these findings as counterevidence to processing-based accounts of island effects. Here, we take up several problems with their methods, their findings, and their conclusions. First, the arguments against processing accounts are based on null results using tasks that may be ineffective or inappropriate measures of working memory (the n-back and serial recall tasks). The authors provide no evidence that these two measures predict judgments for other constructions that are difficult to process and yet are clearly grammatical. They assume that other measures of working memory would have yielded the same result, but provide no justification that they should. We further show that whether a working memory measure relates to judgments of grammatical, hard-to-process sentences depends on how difficult the sentences are. In this light, the stimuli used by the authors present processing difficulties other than the island violations under investigation and may have been particularly hard to process. Second, the Sprouse et al. results are statistically in line with the hypothesis that island sensitivity varies with working memory. Three out of the four island types in their Experiment 1 show a significant relation between memory scores and island sensitivity, but the authors discount these findings on the grounds that the variance accounted for is too small to have much import. This interpretation, however, runs counter to standard practices in linguistics, psycholinguistics, and psychology.∗ Acceptability judgments are inherently ambiguous. Judging a sentence’s acceptability potentially involves the assessment of its syntactic well-formedness, the online processing difficulty encountered while parsing it, as well as other factors (Chomsky & Miller, 1963; Miller & Chomsky, 1963; Bever, 1970; Schütze, 1996; Cowart, 1997; Staum Casasanto et al., 2010; Hofmeister et al., in press). Hence, it is unsurprising that there is disagreement about the explanation for some acceptability contrasts. It is even less surprising when one considers that there are no agreed upon diagnostics distinguishing grammatically-based acceptability contrasts from those based on processing cost. Sprouse et al. (in press; hereafter, SWP) suggest that the relationship between measures of working memory and acceptability judgments may be one such diagnostic. They attempt to use it to resolve a long-standing debate regarding the source of island effects. The term “island effects” here refers to the low acceptability ratings given to sentences with a dependency between a phrase and a syntactic position inside select syntactic environments. For instance, the wh-element, what, in (1) is linked to an argument position inside a complex noun phrase (in brackets): (1) What did Jim repeat [the rumor that Spock loved ]? Ross (1967) lists an array of such syntactic environments that block dependency formation. Since that time, the associated island constraints have been entrenched in linguistic theory as instances of universal grammatical principles and have acted as standard tests for overt and covert phrasal movement (Chomsky, 1973, 1977, 1981, 1986). Numerous researchers, however, have noted that island violations often co-occur with features known to produce processing difficulty (Kuno, 1973; Deane, 1991; Kluender, 1991, 1998, 2005; Kluender & Kutas, 1993; Hofmeister, 2007; Hofmeister et al., 2007; Sag et al., 2007; Hofmeister & Sag, 2010), such as long-distance dependencies spanning multiple new discourse referents, opportunities for garden-pathing or misanalysis, syntactically and semantically similar discourse referents, vague or non-specific filler phrases, etc. For instance, the dependency in (1) not only spans multiple discourse references, but the parser is likely to attempt to integrate the wh-item at the first verb (repeat), leading to reanalysis when the subsequent NP is encountered, or even later. Notably, the island violation is less severe when what is replaced with who, which is explainable on the hypothesis that who is not a particularly plausible argument of repeat. These facts, and others like them, motivate the alternative hypothesis that island effects follow from the increase in processing difficulty that such constructions engender directly and typically co-occur with in examples found in the literature. SWP set out to evaluate such a processing-based theory of islands, what they label a “reductionist” account, by examining how individual differences in processing resources relate to acceptability judgments. The intuition at play is a reasonable one: if judgments of island-violating sentences reflect processing limitations, then individuals with more processing resources should be less likely to “run out” of resources while processing a sentence with an island violation. The acceptability contrast between island-violating and minimally different, non-violating sentences should accordingly be smaller for such individuals than it is for those with fewer language processing resources. To assess the relationship between individual working memory (WM) resources and island sensitivity, SWP calculated differences-in-differences (DD) scores for each participant. These scores are calculated by first subtracting a participant’s mean acceptability judgment for sentences with a dependency into an island (2b) from their mean judgment for sentences with a dependency into an embedded, non-island structure (2a). This difference is termed D1. Next, the participant’s mean judgment for sentences with an island structure but which the wh-dependency does not enter (2d) is subtracted from the mean for sentences with an embedded non-island constituent and a whdependency that does not enter into the constituent (2c). This difference, termed D2, is subtracted from D1 to yield the overall DD score. (2) a. What do you think that John bought? b. What do you wonder whether John bought? c. Who thinks that John bought a car? d. Who wonders whether John bought a car? In essence, these scores reflect “how much greater the effect of an island structure is in a long-distance dependency sentence than in a sentence with a local dependency” (Sprouse et al., in press). The main findings from SWP show that how respondents performed on the memory assessment tasks accounted for only 0-6% of the overall variance in the magnitude of island effects (DD scores). The authors interpret these facts, whose statistical reliability is well-established, as counter to the predictions of a processing-based perspective of island effects. There are several notable obstacles, however, that stand in the way of interpreting the results as the authors do, which we briefly summarize below: The Relationship Between Individual Working Memory Measures and Acceptability Judgments i. The null results leave open the possibility that the authors have selected an inappropriate WM measure, as the authors acknowledge; ii. There is no extant data on how the chosen memory measures relate to acceptability judgments for uncontroversially hard-to-process sentences of English;